feat(usage-maintenance): complete archive, import, and diagnostics workflow - #579

Add archive location and retention settings with validated defaults and environment/file precedence.

Preserve existing data-directory and database-path resolution while keeping automatic retention disabled by default.

Add the stopped-server compact-usage command with process locking, exclusive SQLite access, integrity validation, and before/after summaries.

Prevent concurrent Manager Server access and keep logical usage data unchanged during physical compaction.

…services

Implement archive repositories, gzip JSONL segments, identity-ledger integration, resumable retention workers, coverage-aware readers, and usage import compatibility.

Preserve raw fallback and enforce coverage gates before deletion while allowing startup and restart-safe maintenance progress.

Add admin-only archive preview, create, list, status, resume, verify, delete, and maintenance capability APIs.

Sanitize public DTOs and authorization failures while preserving panel authentication and CORS compatibility.

Add the gated usage maintenance page, archive API client, coverage warning, demo fixtures, routing, layout integration, locale strings, and frontend tests.

Keep CPA-hosted panels isolated and ensure demo state remains resettable and non-persistent.

Document archive backup and restore, gzip JSONL recovery into isolated empty instances, retention, compaction, endpoints, security, and rollback procedures.

Clarify sensitive-data handling and the required operational sequencing for logical deletion and offline physical compaction.

Merge the latest dev branch into the usage maintenance implementation branch.

Preserve the current branding, provider cooling compatibility, demo server metadata, and usage maintenance workflow on one reviewed baseline; no remote state is changed by this commit.

Add stage-bound archive resume handling, raw data range statistics, and a guided maintenance experience with explicit archive, verify, and delete states.

The changes prevent stale concurrent actions from advancing the wrong stage, improve recovery and empty-state guidance, and preserve compatibility with older maintenance payloads; destructive deletion remains separately confirmed.

Add a compatible requested_stage migration and repository APIs for durable, idempotent archive stage scheduling.

Expose filtered keyset archive history with totals and status counts while preserving explicit destructive-stage authorization.

Start durable archive jobs and import-session cleanup after the HTTP listener, and add accepted background archive actions with polling headers.

Add sanitized paginated archive and import-session history contracts, explicit coverage watermarks, lifecycle recovery, and concurrency regression coverage.

Add typed archive filters, cursors, background action options, import-session history capabilities, and coverage watermark DTOs.

Harden runtime response validation and demo fixtures so the existing maintenance UI can safely consume the Phase 0 backend contracts.

Add API-backed overview, filtered archive history, and active/detail presentations that mirror the approved maintenance designs.

Keep progress determinate only with valid totals, preserve explicit destructive actions, and cover polling, pagination, responsive layout, localization, and sanitized segment summaries.

…ation views

Recreate the approved archive creation workflow with retention presets, API-backed impact previews, readiness states, responsive layout, and guided archive verification progress.

Add a structured destructive confirmation that preserves archive semantics, surfaces exact deletion limits, and supports feature-specific modal sizing across themes and breakpoints.

Keep raw deletion independently confirmed and avoid presenting source-row estimates as compressed archive sizes.

Move import file detection into a shared utility while preserving the monitoring model export.

This removes a cross-feature model dependency and keeps maintenance and monitoring validation behavior identical.
No runtime data or API behavior changes are introduced.

Allow usage import orchestration to resume a selected Manager Server session and validate the file identity before uploading.

Preserve local resume-key compatibility, map explicit mismatches to a safe conflict code, and cover normalized filenames plus rejection paths with focused tests.

Implement the HTML-aligned import/export maintenance view with resumable chunk sessions, sanitized JSONL export, status polling, pause/resume/cancel actions, and safe result summaries.

Add responsive dark/light themed styling, accessible confirmation and file-picker flows, localized copy in all supported locales, and focused UI coverage for export, import confirmation, safe errors, retryability, and unmount cancellation.

Add offline-only SQLite compaction guidance, complete backup requirements, and server-backed coverage and lock diagnostics while preserving sanitized API boundaries.

Wire overview navigation, refresh error handling, clipboard fallback, responsive styles, and four-locale translations with regression coverage.

Remove duplicated adaptUsageAnalyticsData import specifier in
usageAnalyticsModel.test.ts introduced during merge resolution.
Restores type-check cleanliness without impacting runtime logic.

…odex evidence gate

Include auth_account_id_snapshot in archived JSON records and retain it
during event import to ensure distinct member identities in shared workspaces
survive archive round-trips without identity collapse.
Enforce codex_legacy_identity_v1 readiness and correct projection revision
in delete validation before allowing raw usage event deletions.

…baseline

Restore strict structure revision validation for hourly usage aggregate.
Accept only the base revision ('schema-3:model-1') and valid 32-hex rebuild
revisions while rejecting deprecated cache-accounting-v2 suffixes.
Eliminates circular package dependency between sqlite and usageaggregate.

…ty evidence reader

Calculate latestKnownID using max(targetEventID, rawMaxID) so stored codex
evidence remains valid and readable even after historical raw events are deleted
(max(id) = 0). Incoming tail events continue to trigger bounded tail scans.

…w events are deleted

Add ensureCompleteRawSourceForDerivedRebuild guard across derived table rebuild
paths. Reject destructive schema rebuilding and state clearing if historical raw
events were previously archived and deleted, keeping existing derived data intact.

…er compat and retention tests

Include CatchUpCodexLegacyIdentityEvidence in catchUpCompatUsageDerived and
catchUpRetentionWorkerReadiness helper loops to satisfy the new delete readiness gate.
Ensures server compat lifecycle and retention worker restart tests pass cleanly.

…aw deletion

Add HistoricalRawDeletionExists helper to check for historical raw event
deletions recorded in usage_raw_deletion_audit. Guard both
resetDamagedUsageMonitoringDerivations and resetUsageDerivedDataWithoutSource
in SQLite migration routines right after detecting damage and before running
any destructive SQL deletes.

Ensures that when raw source events have been deleted, damaged migration
recovery fails closed with ErrDestructiveRebuildDisallowed rather than
rebuilding incomplete aggregates.

…ss catch-up

Remove the erroneous condition in EnsureEvidenceRollup that wiped out
incremental Codex legacy identity rollup state when latestID was behind
the recorded state.CoverageEventID.

Allows the evidence reader to incrementally catch up with newly inserted raw
events after historical pruning without losing previously accumulated legacy
identity evidence.

…fy round-trip identity

Decouple Codex legacy identity evidence from usagemonitoring.SchemaVersion
by allowing stateRequirement to specify an explicit expectedVersion
(usageevent.CodexLegacyIdentityEvidenceSchemaVersion).

Add integration tests verifying that auth_account_id_snapshot and member
email distinctions survive full archive extraction, raw purging, and
restoration cycles without identity key collisions.

…force fail-closed readiness

Extract collectResponseMetadataBackfillUpdates to ensure rows matching SQL predicates but yielding no JSON changes are not treated as pending backfill.

Enforce fail-closed check in ensureManualArchiveReadiness and remove the live backfill loop from manual archive execution.

…and enforce schema completeness

Disallow cancelling archive runs that have already published segments (archived_event_count > 0) in both repository and UI model with ErrCancelPublished.

Add comprehensive archive schema contract tests covering every column of usage_events.

…flight

Include the resumable archive import header X-Usage-Import-Prefix-SHA256 in Access-Control-Allow-Headers across CORS preflight responses.

Add unit test in cors_test.go verifying the header is permitted.

…ked by raw deletion

Map ErrStructureChangeAfterRawDeletion to HTTP 409 Conflict with error code model_price_structure_locked_by_usage_archive for PUT and Sync endpoints.

Export frontend error code constant and add comprehensive HTTP handler test coverage.

… version from aggregate schema

Introduce CodexLegacyIdentityEvidenceSchemaVersion = 1 to decouple codex evidence payload verification from aggregate roll-up schema version bumps.

Add unit tests verifying schema decoupling and legacy roll-up persistence across migrations.

…reeze upon raw event deletion

Update deletion confirmation dialog across en, zh-CN, zh-TW, and ru locales to explicitly warn that raw deletion permanently locks historical model prices and freezes destructive rebuilds, recommending a complete pre-deletion backup.

Update unit test assertions in UsageMaintenancePage.test.tsx.

…er raw usage deletion

Clarify in backup and configuration guides (zh and en) that deleting historical raw usage events permanently freezes destructive rebuilds and locks model price structures.

Highlight the necessity of taking a full backup prior to raw event deletion.

…hive creation

Organize usage maintenance into persistent Organize data, Import/export
and Processing records tabs. Keep purpose selection, date range preview,
archiving, verification and separately confirmed cleanup in one workflow.

Persist navigation in the URL and restore record details, focus and
scroll. Preserve valid results across cancellation, late callbacks and
summary refresh failures. Share import progress with Monitoring and keep
confirmation actions visible on mobile.

Update four locales and both operations guides while preserving
archive-wide cleanup, deletion gates and offline compaction boundaries.

Validation:
- Frontend and repository suites: 3,763 tests passed.
- Drawer/navigation coverage: 78 tests passed, including a regression
  verified to fail before the record-reopening fix.
- Final notification translations: key/placeholder checks and 11 import
  tests passed after the full suite.
- TypeScript, Demo, Docker production and documentation builds passed.
- Lint: no errors and five pre-existing Accounts warnings.
- 60 locale/theme/viewport layouts and isolated Demo workflows passed.

Final real-site browser checks remain incomplete after approval service
timeouts and a Playwright transport failure. Final served resources were
verified over HTTP; Demo workflows do not verify real database recovery.